This invention relates to an improved neural network circuit for image recognition processing or the like.
Neural networks have been noted in recent years in the field of data processing which are modeled on the neural signal processing of living systems of a human body. Various neural networks have been proposed currently. Among them, a network of quantized neurons is known. Image recognition by means of such a network is disclosed in the papers, "Character Recognition System Using Network Comprising Layers By Function (Preliminary Material for Image Electronic Communication Society's National Conference 1990, pages 77-80) and "Multi-Functional Layered Network Using Quantizer Neurons (Computer World '90, November).
FIG. 21 shows a quantized neuron network. This network comprises first (lowermost) to fifth (uppermost) layers. The quantized neurons of the first, second and third layers designated by circles are interconnected in a multi-layer network-like form. The fourth layer serves as a supervisor input layer. The fifth layer, or last layer comprises conventional artificial neurons of a "multi-input-and-single-output" type. Characteristic data are supplied to the first to third layers respectively. For example, 64 pixel values of 8.times.8 segments of an image, lateral differential values among the 64 pixel values, and vertical differential values among the 64 pixel values are supplied to these layers. As shown in FIG. 21, the first layer is made up of 64 quantized neurons corresponding to the number of pixels (=8.times.8 pixels). As shown in FIG. 22, the quantized neurons of the first layer are provided with quantization signal input terminals S through which the pixel values, serving as characteristic data, are supplied. The quantization signal input terminals S of the quantized neurons of the second layer are supplied with the lateral differential values. The quantization signal input terminals S of the quantized neurons of the third layer are supplied with the vertical differential values. The fourth layer (i.e., the supervisor input layer) modifies the coupling coefficient of a synapse between neurons of the last layer by means of supervisor input. The last layer is a solution output layer. If such a solution is for 62 alphanumeric characters, the last layer will comprise 62 conventional artificial neurons of "multi-input-and-single-output" type.
As shown in FIG. 25, in the neuron of the output layer, each data input thereto is multiplied by a weight Wn ("n" represents an input number) determined through learning, and the sum of the results (products) is output.
As shown in FIG. 22, the quantized neurons (quantized cells) of the first to third layers further include selective signal input terminals R and a plurality of outputs, in addition to the quantization signal input terminals S. A signal input through the selective signal input terminal R is multiplied by each of weights .tau. according to Formula (1), by an input value x supplied from the quantization signal input terminal S, and then the results are output to the respective outputs. EQU .tau.j=1-.beta.(.vertline.j-x.vertline.) (1),
where .vertline.j-x.vertline.=absolute value of j-x; j=quantized neuron output number.
For the quantized neuron of FIG. 22, the number of input levels of the characteristic data supplied through the quantization signal input terminal S is eight, and the number of outputs thereof is eight. For instance, supposing that the function of (1-.beta.) is one as shown in FIG. 24, the value that will appear at each of the eight outputs of the quantized neuron can be found by multiplying the value supplied to the selective signal input terminal R by the coupling coefficient of FIG. 23(a), when the value supplied from the quantization signal input terminal S is 0. Irrespective of the values fed to the selective signal input terminal R, if the input value of the quantization signal input terminal S is 3, each coupling coefficient will be obtained by sliding each of the coupling coefficients of FIG. 23(a) three positions to the right, as shown in FIG. 23(b). In other words, in the event that the input value of the quantization signal=3, each coupling coefficient can be found by adding 3 to each output number of FIG. 23(a). However, in the event that the sum goes beyond 7, an additional subtraction (the sum-8) is required.
The input supplied to the selective signal input terminal R of the quantized neuron of the first layer is given a maximum value. Supposing that 8-bit arithmetical operations are performed in the quantized neuron, that the coupling coefficients of the first layer are ones as shown in FIG. 23(a), and that the level of a value given to the quantization signal input terminal S is 0, the following are found, that is, the selective signal input=FF, the output number 0=FF, the output number 1=7F, the output number 2=0, the output number 3=0, the output number 4=0, the output number 5=0, the output number 6=0, and the output number 7=7F (hexadecimal notation).
For the network made up of quantized neurons of FIG. 22 each having the quantization signal input terminal S to which the input are supplied at different eight levels, and the eight outputs, the eight outputs of the quantized neuron of the first layer are connected to the selective signal input terminals R of the quantized neurons of the second layer. Similarly, the eight outputs of the quantized neuron of the second layer are connected to the selective signal input terminals R of the quantized neurons of the third layer. In this network, a single quantized neuron of the first layer branches off upwardly in a tree-like form through the second layer to the third layer. The number of outputs of the quantized neurons of the third layer totals up to 512.
Eight quantized neurons of the second layer connected to the eight outputs of the quantized neuron of the first layer, and eight quantized neurons of the third layer connected to the eight outputs of the quantized neuron of the second layer are supplied with, as quantization signal input, lateral differential values and vertical differential values, respectively, both the lateral differential values and vertical differential values locating at the same positions as the pixel values fed to the quantization signal input terminals S of the quantized neurons of the first layer as characteristic data, respectively.
The signals to be input to the selective signal input terminals R of the quantized neurons of the second layer are the outputs of the quantized neurons of the first layer. The signals to be input to the selective signal input terminals R of the quantized neurons of the third layer are the outputs of the quantized neurons of the second layer.
Therefore, the output of the quantized neuron of the third layer is obtained by a multiplication (that is, the coupling coefficient of the neuron of the first layer.times.the coupling coefficient of the neuron of the second layer.times.the coupling coefficient of the neuron of the third layer.times.the input value of the selective signal of the quantized neuron of the first layer).
A single neuron of a "multi-input-and-single-output" type of the last layer is supplied with a product of 64.times.512 outputs of the quantized neurons of the third layer times the coupling coefficients of the fourth layer (supervisor input layer) given to these outputs respectively.
Many of the arithmetical operations in a neural network are realized by conventional yon Neumann-type computers. For example, since the amount of operation of the network of FIG. 21 per input is expressed by Formula (2):
Multiplication : 8.times.8.times.8=512 PA1 Product-Sum Operation: 8.times.8.times.8.times.62=31744 (2) PA1 Formula (3) holds for an input image of 8.times.8 segments. PA1 Multiplication : 512.times.64=32768 PA1 Product-Sum Operation: 31744.times.64=2031616 (3) PA1 However, many of the neuro-computers carry out every network arithmetical operation with the help of digital computers of yon Neumann-type in a sequential, serial manner, as a result of which a high processing speed may not be achieved if the amount of operation is great.
There is a method for accomplishing high processing speed which utilizes many operation units which operate in parallel processing to simultaneously perform a plurality of arithmetical operations, however, such a method requires very large-scale circuit which is costly.